Display device

ABSTRACT

A display device configured to be used in a state of being fixed to a head of a user, includes: a camera; a display configured to display a part of an image captured by the camera as a display range; an orientation detection sensor configured to detect an orientation of the display device; a line-of-sight detection sensor configured to detect a line-of-sight of the user to the display; and a processor configured not to control a position of the display range based on the line-of-sight detected by the line-of-sight detection sensor in a case where a change amount of the orientation detected by the orientation detection sensor is smaller than a predetermined amount, and to control the position of the display range based on the line-of-sight in a case where the change amount of the orientation is larger than the predetermined amount.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Continuation of International Patent ApplicationNo. PCT/JP2020/029990, filed Aug. 5, 2020, which claims the benefit ofJapanese Patent Application No. 2019-205442, filed Nov. 13, 2019, andJapanese Patent Application No. 2019-205493, filed Nov. 13, 2019, whichare hereby incorporated by reference herein in their entirety.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to a display device, and more particularlyto an electronic binocular telescope.

Background Art

An electronic binocular telescope is a display device that includes acamera and displays which are disposed in front of the eyes of a userwhen the electronic binocular telescope is in use, and displays imagescaptured by the camera on the displays in real-time. By looking at theimages (telescopic images) displayed on the displays of the electronicbinocular telescope, the user can observe a distant area as if lookingthrough binoculars. Some electronic binocular telescopes are configuredto be detachably mounted (wearable) on the head, such as a case of ahead mounted display.

PTL 1 discloses a technique to detect a line-of-sight position of theuser, and control an image-capturing range so that the line-of-sightposition comes to the center position of the image-capturing range.

When the electronic binocular telescope is used, it is preferable toprovide a field-of-view to the user in a same way as observing with thenaked eye. In the case of observing with the naked eye, thefield-of-view expands in the direction of the line-of-sight, regardlessthe movement of the head and the orientation (direction) of the face.However, in the case of the commonly used electronic binoculartelescope, the field-of-view (object range (angle-of-view) of the imagesdisplayed on the displays; observation range) changes in accordance withthe movement of the head, regardless the direction of the line-of-sight.Since the observation range changes by an unintentional movement of thehead, the user has a sense of discomfort and has difficultly to focus onthe observation. Particularly when a highly magnified image isdisplayed, the observation range changes considerably by a slightmovement of the head.

In a case of using the technique disclosed in PTL 1, in which theimage-capturing range becomes the observation range, the line-of-sightposition becomes the center position of the image-capturing range, hencethe observation range changes depending on the line-of-sight, even ifthe user wants to look out over a desired observation range.

In this way, in the prior arts, the observation range may beunintentionally changed, and the user may experience a sensationcompletely different from observation with the naked eye (sense ofdiscomfort). This problem occurs in both a case of a wearable electronicbinocular telescope and a case of an electronic binocular telescope thatis not wearable.

The present invention provides a technique with which an unintentionalchange of the observation range can be minimized, and the user canexperience a sensation close to the observation with the naked eye(sensation with no or minimal sense of discomfort).

CITATION LIST Patent Literature

PTL 1 Japanese Patent Application Laid-open No. 2015-149552

SUMMARY OF THE INVENTION

The present invention in its first aspect provides a display deviceconfigured to be used in a state of being fixed to a head of a user,including: a camera; a display configured to display a part of an imagecaptured by the camera as a display range; an orientation detectionsensor configured to detect an orientation of the display device; aline-of-sight detection sensor configured to detect a line-of-sight ofthe user to the display; and a processor configured not to control aposition of the display range based on the line-of-sight detected by theline-of-sight detection sensor in a case where a change amount of theorientation detected by the orientation detection sensor is smaller thana predetermined amount, and to control the position of the display rangebased on the line-of-sight in a case where the change amount of theorientation is larger than the predetermined amount.

The present invention in its second aspect provides a control method ofa display device including a camera and a display configured to displaya part of an image captured by the camera as a display range, thedisplay device being configured to be used in a state of being fixed toa head of a user, the control method including: an orientation detectionstep of detecting an orientation of the display device; a line-of-sightdetection step of detecting a line-of-sight of the user to the display;and a control step of not controlling a position of the display rangebased on the line-of-sight detected in the line-of-sight detection stepin a case where a change amount of the orientation detected in theorientation detection step is smaller than a predetermined amount, andcontrolling the position of the display range based on the line-of-sightin a case where the change amount of the orientation is larger than thepredetermined amount.

The present invention in its third aspect provides a non-transitorycomputer readable storage medium that stores a program, wherein theprogram causes a computer to execute a control method of a displaydevice including a camera and a display configured to display a part ofan image captured by the camera as a display range, the display devicebeing configured to be used in a state of being fixed to a head of auser, the control method including: an orientation detection step ofdetecting an orientation of the display device; a line-of-sightdetection step of detecting a line-of-sight of the user to the display;and a control step of not controlling a position of the display rangebased on the line-of-sight detected in the line-of-sight detection stepin a case where a change amount of the orientation detected in theorientation detection step is smaller than a predetermined amount, andcontrolling the position of the display range based on the line-of-sightin a case where the change amount of the orientation is larger than thepredetermined amount.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a flow chart depicting a processing flow according toEmbodiment 1;

FIGS. 2A and 2B are external views of an electronic binocular telescopeaccording to Embodiments 1 to 4;

FIG. 3 is a block diagram of the electronic binocular telescopeaccording to Embodiments 1 to 4;

FIGS. 4A to 4D are schematic diagrams depicting states of a displaydevice and a user according to Embodiments 1 and 2;

FIG. 5 is a schematic diagram depicting a relationship between an objectdistance and a moving amount according to Embodiments 1 and 2;

FIG. 6 is a flow chart depicting a processing flow according toEmbodiment 2;

FIG. 7 is a flow chart depicting a processing flow according toEmbodiment 3;

FIGS. 8A to 8D are schematic diagrams according to the states of adisplay device and a user according to Embodiments 3 and 4;

FIG. 9 is a schematic diagram depicting a relationship between an objectdistance and a rotating amount according to Embodiments 3 and 4; and

FIG. 10 is a flow chart depicting a processing flow according toEmbodiment 4.

DESCRIPTION OF THE EMBODIMENTS Embodiment 1

Embodiment 1 of the present invention will be described. Here an exampleof applying the present invention to an electronic binocular telescopethat can be detachably mounted (wearable) on the head such as a headmounted display, will be described, but a display device to which thepresent invention is applicable is not limited to the wearableelectronic binocular telescope. For example, the present invention maybe applied to an electronic binocular telescope that is not wearable, orthe present invention may be applied to other wearable devices that canbe detachably mounted on the head (e.g. head mounted displays of VRgoggles, AR glasses, MR glasses, smart glasses, or the like). Thepresent invention may also be applied to a display device covering botheyes (e.g. VR goggles) such that the user cannot view the surroundingswith the naked eye (the user can view the image with both eyes).Furthermore, the present invention may be applied to a display devicethat covers only one eye, so that the user can view an image using theone eye and view the surroundings with the naked eye using the othereye. According to the present invention, good observation becomespossible even in a state where the user cannot view the surroundingswith the naked eye, hence the present invention is suitable for adisplay device that covers both eyes (details described later).

FIGS. 2A and 2B are external views of a hardware configuration of anelectronic binocular telescope 10 according to Embodiment 1. FIG. 2A isa front perspective view when the electronic binocular telescope 10 isviewed from the front, and FIG. 2B is a rear perspective view when theelectronic binocular telescope 10 is viewed from the back. Theelectronic binocular telescope 10 is a spectacle type electronicbinocular telescope, and can be detachably mounted on the head.Specifically, the electronic binocular telescope 10 can be fixed(mounted) on the head by cradling the head between the left and righttemples 100 of the electronic binocular telescope 10. Besides thetemples 100, the electronic binocular telescope 10 includes a camera101, displays 102 and 103, a panning unit 104, a tilting unit 105, agyro sensor 106, a line-of-sight-detecting unit 107 and an operationmember 108.

The camera 101 is an imaging unit and can be rotated in the pandirection and the tile direction independently as illustrated by arrowsin FIG. 2A. In other words, the imaging direction of the camera 101 (adirection from the camera 101 to an object in the imaging range, such asa direction from the camera 101 to an object position which correspondsto the center position of the imaging range; optical axis direction ofthe camera 101) can be changed in the pan direction and the tiltdirection independently. The panning unit 104 rotates the camera 101 inthe pan direction (direction to tile the camera 101 to the left or rightwith respect to the electronic binocular telescope 10) by driving anactuator included in the panning unit 104. The tilting unit 105 rotatesthe camera 101 in the tilt direction (a direction to tilt the camera 101up or down with respect to the electronic binocular telescope 10) bydriving the actuator included in the tilting unit 105. The changingdirection, mechanism and the like, to change the imaging direction, arenot especially limited.

The camera 101 is configured such that the focal distance thereof can bechanged. In Embodiment 1, the focal distance of the camera 101 can beswitched between 100 mm and 400 mm in two stages (both focal distanceshave been converted into full size 35 mm) in accordance with theoperation the user performed on the electronic binocular telescope 10(user operation). The operation member 108 is a member (e.g. button,switch) to receive user operation, and receives a user operation toinstruct the change (switch) of the focal distance of the camera 101, orturning the power of the electronic binocular telescope 10 ON/OFF, forexample. A number of focal distances and the range thereof that can beset are not especially limited. The focal distance may be changeableseamlessly within a predetermined range.

The camera 101 has an auto focus function and is configured toautomatically focus on an object included in the imaging range.Depending on the stopping position of a focusing lens (not illustrated),which is driven during focus adjustment (auto focus), an object distanceat which the object is focused is uniquely determined. Therefore ifinformation (tables and functions) that indicates the relationshipbetween the stopping position of the focusing lens and the objectdistance is stored in the electronic binocular telescope 10 in advance,the electronic binocular telescope 10 can detect the object distancebased on the stopping position of the focusing lens using thisinformation. The camera 101 also includes the function to detect theobject distance using this method. However, the method for detecting theobject distance is not especially limited.

The displays 102 and 103 are display units that display a part of theimage captured by the camera 101 as a display range. The display rangemay be displayed based on the image generated by developing the entireimageable range, or only the display range may be read from the camera101 (image pickup element), and developed and displayed. When the userwears the electronic binocular telescope 10, the display 102 is disposedin front of the right eye of the user, and the display 103 is disposedin front of the left eye of the user. This means that the user views theimage display on the display 102 using the right eye, and views theimage displayed on the display 103 using the left eye. The display rangecan be moved in the pan direction (left-right direction (horizontaldirection) of the captured image) and the tilt direction (up-downdirection (vertical direction) of the captured image) independently. Themoving direction of the display range (direction of changing theposition of the display range) is not especially limited.

The gyro sensor 106 is an orientation-detecting unit that detects theorientation of the electronic binocular telescope 10, and can alsodetect the change of the orientation (e.g. whether change occurred ornot, direction of the change, magnitude of the change) of the electronicbinocular telescope 10. In the case where the user is wearing theelectronic binocular telescope 10, the orientation of the electronicbinocular telescope 10 corresponds to the orientation of the head of theuser. Therefore the gyro sensor 106 can detect the orientation andmovement (e.g. shaking) of the head.

The line-of-sight-detecting unit (line-of-sight-detection sensor) 107detects the line-of-sight (relative line-of-sight) of the user to thedisplays 102 and 103. The line-of-sight-detecting unit 107 can alsodetect the changes (e.g. whether change occurred or not, direction ofthe change, magnitude of the change) of the line-of-sight.

FIG. 3 is a block diagram depicting a configuration of the electronicbinocular telescope 10. A CPU 201 controls each component of theelectronic binocular telescope 10. The CPU 201 is connected to thecamera 101, the displays 102 and 103, the gyro sensor 106, theline-of-sight-detecting unit 107, the operation member 108 and acamera-rotating unit 202, and the like. The CPU 201 processesinformation from each component of the electronic binocular telescope10, and controls the operation of each component in accordance with theprocessing result. The camera-rotating unit 202 includes the panningunit 104 and the tilting unit 105, and rotates the camera 101 in the pandirection or the tilt direction in accordance with the instruction fromthe CPU 201.

FIG. 1 is a flow chart depicting a processing flow (processing flow ofthe electronic binocular telescope 10) according to Embodiment 1. Theprocessing flow in FIG. 1 is implemented, for example, by the CPU 201developing a program, stored in ROM (not illustrated), in RAM (notillustrated), and executing the program. When the user operation, toinstruct to turn the power of the electronic binocular telescope 10 ON,is performed, the electronic binocular telescope 10 starts up, andprocessing to display a part of the image, captured by the camera 101 asa display range on the displays (displays 102 and 103) in real-time, isstarted. Thereby the user can view the image captured by the camera 101displayed on the display, and start observing the object image. Then theprocessing flow in FIG. 1 starts. The initial value of the focaldistance (focal distance immediately after power is turned ON) of thecamera 101 is not especially limited, but is preferably a wide angelfocal distance so that the user can easily find an observation target.In Embodiment 1, the focal distance is controlled to 100 mm immediatelyafter the power is turned ON. The focal distance is switched between 100mm and 400 mm, each time the user operation to instruct to change(switch) the focal distance is performed during the processing flow inFIG. 1 . (This is not indicated in FIG. 1 .) Further, in Embodiment 1,it is assumed that the camera 101 is fixed so that the optical axis ofthe camera 101 is parallel with the front face direction of theelectronic binocular telescope 10 (direction to which the face of theuser wearing the electronic binocular telescope 10 is facing).

FIG. 4A indicates the initial direction (display direction immediatelyafter power is turned ON; reference direction) of the display direction(direction from the camera 101 to the object in the display range, suchas direction from the camera 101 to the object position corresponding tothe center position of the display range). As illustrated in FIG. 4A,the reference direction is a direction matching with the optical axis ofthe camera 101 and is a direction parallel with the front face directionof the electronic binocular telescope 10 (direction in which the face ofthe user wearing the electronic binocular telescope 10 is facing). FIG.4A is drawn from the viewpoint of viewing the head of the user fromabove, so that the pan direction component of the display direction canbe visually understood, but this is the same for the tilt directioncomponent of the display direction as well. In the following, only thecontrol to change the display direction (position of the display range)in the pan direction will be described, but the display direction mayalso be changed in the tilt direction by the same control method as thecontrol method to change the display direction in the pan direction.

In S101 in FIG. 1 , the camera 101 performs auto focus (AF) control andauto exposure (AE) control based on the captured image.

In S102, the camera 101 detects (acquires) the object distance L fromthe result of the AF control in S101, and the CPU 201 determines(calculates) the moving amount A1 of the display range based on thedetected object distance L.

FIG. 5 indicates the relationship between the object distance L and themoving amount A1. A star symbol in FIG. 5 indicates the observationtarget that exists in front of the user. Normally in the case ofobservation with the naked eye, the user faces the object and capturesthe object at the center of the field-of-view. Here a case where thedisplay direction is the reference direction will be considered. In thiscase, depending on the position at which the camera 101 is installed,the observation target, which the user would capture at the center ofthe field-of-view if the user were viewing with the naked eye, may notbe displayed at the center of the display, and the user may feel a senseof discomfort. The moving amount A1 determined in S102 is a movingamount to reduce such a sense of discomfort. Here it is assumed that,moving direction to the left is a positive direction, and a movingdirection to the right is a negative direction. In FIG. 5 , the camera101 is installed at a position that is shifted to the right side fromthe center of the head by the distance a. Therefore if the display rangeis moved by a moving amount A1=φ1=arctan (a/L1), the observation targetexisting at the object distance L1 (observation target existing in frontof the user) can be displayed at the center of the display. In the samemanner, if the distance range is moved by a moving amount A1=φ2=arctan(a/L2), the observation target existing at the object distance L2(observation target existing in front of the user) can be displayed atthe center of the display. In this way, in S102, the moving amount A1,which is longer as the object distance is shorter, is determined basedon the relational expression “A1=arctan (a/L)” from the object distanceL. According to this relational expression, the moving amount A1 becomesvirtually 0 (zero) when the object distance L is relatively long.Therefore in a case where observation is basically performed only for anobject at long distance, or in a case where the object distance L islonger than a predetermined distance, the moving amount A1 may be set to0 (moving amount A1=0).

In S103 in FIG. 1 , the CPU 201 determines whether the focal distance ofthe camera 101 is longer than a threshold (predetermined distance).Processing advances to S104 if the focal distance is longer than thethreshold, and processing advances to S109 if the focal distance is thethreshold or less. It is also acceptable that processing advances toS104 if the focal distance is the threshold or more, and processingadvances to S109 if the focal distance is less than the threshold. InEmbodiment 1, the focal distances that can be set are 100 mm and 400 mm,hence in S103, it is determined whether the focal distance is 400 mm ornot, and processing advances to S104 if the focal distance is 400 mm,and processing advances to S109 if the focal distance is 100 mm.

In S104, the CPU 201 detects the orientation of the electronic binoculartelescope 10 (head) using the gyro sensor 106, and determines whetherthe orientation of the electronic binocular telescope 10 (head) changedby a change amount larger than a threshold (predetermined amount).Processing advances to S105 if the orientation changed by a changeamount lager than the threshold, and processing advances to S109 if thechange amount of the orientation is the threshold or less. It is alsoacceptable that processing advances to S105 if the orientation changedby the threshold or a larger change amount, and processing advances toS109 if the change amount of the orientation is less than the threshold.

In S105, using the line-of-sight-detecting unit 107, the CPU 201 detectsthe line-of-sight of the user in a period when the orientation of theelectronic binocular telescope 10 (head) is changing, and switchesprocessing so that the position of the display range is controlled basedon this line-of-sight.

FIG. 4B indicates a state where the head of the user unintentionallymoved in the state of FIG. 4A. In the case where the head movesunintentionally, the absolute line-of-sight is more likely to bemaintained in an approximate constant state. Therefore in FIG. 4B, thehead moved to the left (positive direction) by the angle θ1, but thedirection of the absolute line-of-sight remains at the originalreference direction (reference direction before the head moved;reference direction in FIG. 4A). In other words, the movement to theright (negative direction; opposite direction of the movement of thehead) by the angle θ1 is generated as a relative movement of theline-of-sight with respect to the display. When the head moves, theelectronic binocular telescope 10 moves integrally with the head.Therefore in FIG. 4B, the electronic binocular telescope 10 also movesto the left by the angle θ1. In the state of FIG. 4B, using the gyrosensor 106, the CPU 201 determines that the electronic binoculartelescope 10 (head) moved to the left by the angle θ1. Furthermore,using the line-of-sight-detecting unit 107, the CPU 201 determines thatthe line-of-sight (relative line-of-sight) moved to the right (oppositedirection of the movement of the electronic binocular telescope 10(head)) by the angle θ1.

In the case where the line-of-sight (relative line-of-sight) changed tothe opposite direction of the change direction of the orientation of theelectronic binocular telescope 10 (head) like this, processing advancesto S106 based on the determination that the user does not desire tochange the viewing direction (line-of-sight direction) and the headunintentionally moved. Then in S106, the CPU 201 determines (calculates)the moving amount A2 to move the display range such that theangle-of-view immediately before the detection of the movement of thehead is maintained. In other words, the CPU 201 determines the movingamount A2 to move the display range such that the movement of the headis cancelled. In the state of FIG. 4B, the moving amount A2 to rotatethe display direction to the right by the angle θ1 is determined. Atthis time, the moving amount A2 may be determined based on the detectionresult by the gyro sensor 106 (angle θ1 in the left direction), or maybe determined based on the detection result by theline-of-sight-detecting unit 107 (angle θ1 in the right direction), ormay be determined based on both of these detection results. The angle(degree) detected by the gyro sensor 106 and the angle (degree) detectedby the line-of-sight-detecting unit 107 may be different.

In the example described above, the moving amount A2, to make the changeof the display direction caused by the change of the orientation of theelectronic binocular telescope 10 (head) to 0 (zero), is determined byS106, but the moving amount A2 determined in S106 is not limited tothis. All that is required here is to decrease the change of the displaydirection caused by the change of the orientation, and the change of thedisplay direction need not be exactly 0 (zero).

FIG. 4C indicates a state where the user intentionally moved their headin the state of FIG. 4A in order to change the observation target. Inthe case of intentionally moving the head, the absolute line-of-sight ismore likely to move in the same direction as the moving direction of thehead. Therefore in FIG. 4C, an integrated movement of the head and theabsolute line-of-sight occurred. Specifically, the head moves to theleft by the angle θ1, and the absolute line-of-sight also moves to theleft by the angle θ1. In other words, no movement of the line-of-sightrelative to the display is generated. In the state of FIG. 4C, using thegyro sensor 106, the CPU 201 determines that the electronic binoculartelescope 10 (head) moved to the left by the angle θ1. Furthermore,using the line-of-sight-detecting unit 107, the CPU 201 determines thatthe line-of-sight (relative line-of-sight) is not moved.

In the case where only the change of the orientation of the electronicbinocular telescope 10 (head) is detected like this, processing advancesto S107 based on the determination that the user intentionally movedtheir head in order to change the viewing direction in accordance withthe movement of the head. Then in S107, the CPU 201 determines themoving amount A2=0 which does not move the display range. In otherwords, the CPU 201 determines the moving amount A2 to maintain thedisplay direction in the line-of-sight direction (direction of theline-of-sight).

FIG. 4D indicates a state where the user intentionally moved their headin the state of FIG. 4A, in order to track (visually follow) anobservation target (moving object) that moves fast. In the case oftracking an observation target that moves fast, the absoluteline-of-sight is more likely to move by a moving amount larger than themoving amount of the head (including the moving amount of the head) inthe same direction as the moving direction of the head. Therefore inFIG. 4D, the head moves to the left by the angle θ1, and the absoluteline-of-sight moves to the left by the angle θ1+θ2. In other words, therelative movement of the line-of-sight with respect to the display isgenerated to the left by the angle θ2. In the state of FIG. 4D, usingthe gyro sensor 106, the CPU 201 determines that the electronicbinocular telescope 10 (head) moved to the left by the angle θ1.Furthermore, using the line-of-sight-detecting unit 107, the CPU 201determines that the line-of-sight (relative line-of-sight) moved to theleft by the angle θ2.

In the case where the line-of-sight (relative line-of-sight) changed inthe same direction as the change direction of the orientation of theelectronic binocular telescope 10 (head) like this, processing advancesto S108 based on the determination that the user intentionally movedtheir head in order to radically change the viewing direction. Then inS108, the CPU 201 determines the moving amount A2 to change the displaydirection to the line-of-sight direction.

The moving amount A2 that is determined in S108 is not limited to themoving amount to change the display direction to the line-of-sightdirection. All that is required here is to increase the change of thedisplay direction caused by the change of the orientation, and thedisplay direction need not match with the line-of-sight direction.

In S109, the CPU 201 determines the moving amount A2 as 0 (A2=0). In acase where the focal distance is short (in the case of a wideangle-of-view; corresponds to NO in S103), the change of the observationrange (object range (angle-of-view) of an image displayed on thedisplay) caused by unintentional change of orientation (unintentionalmovement of the head) of the electronic binocular telescope 10 normallydoes not stand out very much. In a case where the change amount of theorientation of the electronic binocular telescope 10 (head) is small(corresponds to NO in S104), the change of the observation range causedby the unintentional change of orientation of the electronic binoculartelescope 10 does not stand out very much, and the user may move theline-of-sight so as to look out over the observation range. Therefore inEmbodiment 1, processing can advance to S109 in such cases. At thistime, the line-of-sight detection is not performed, or even if theline-of-sight detection is performed, the detection result thereof isnot used. When the focal distance is long (focal distance of telephotoobservation; focal distance at which the change of the observation rangecaused by unintentional change of orientation of the electronicbinocular telescope 10 tends to stand out), the processing in S103 maybe omitted so that processing can advance to 5104 regardless the focaldistance.

In S110, the CPU 201 determines (calculates) the final moving amountA=A1+A2 of the display range, based on the moving amount A1 determinedin S102 and the moving amount A2 determined in any step of S106 to S109.In the case where it is predetermined that the moving amount A1 is 0(A1=0), such as a case where the observation is basically performed onthe objects at long distance, the processing in S102 may be omitted sothat the moving amount A=A2 is determined regardless the object distanceL.

In S111, the CPU 201 moves the display range by the moving amount Adetermined in S110.

In S112, the CPU 201 monitors information from the operation member 108,and determines whether the user operation, to instruct to turn the powerof the electronic binocular telescope 10 OFF, was performed. Theprocessing steps S101 to S111 are repeated until the power OFF isinstructed, and if a power OFF is instructed, the processing flow inFIG. 1 ends, and the electronic binocular telescope 10 stops (power ofthe electronic binocular telescope 10 is shut OFF).

As described above, in Embodiment 1, the position of the display rangeis not controlled based on the detected line-of-sight if the changeamount of the detected orientation (orientation of the electronicbinocular telescope 10 (head)) is smaller than a predetermined amount.The position of the display range is controlled based on the detectedline-of-sight if the change amount of the detected orientation is largerthan the predetermined amount. Thereby an unintentional change of theobservation range can be minimized, and a sensation close to theobservation with the naked eye (sensation with no or minimal sense ofdiscomfort) can be provided to the user.

Besides the control of the position of the display range in theprocessing flow of FIG. 1 , the position of the display range may beconstantly controlled based on the shaking of the electronic binoculartelescope 10, such as camera shake correction. Specifically, theposition of the display range may be constantly controlled so that theshaking in the display direction caused by the shaking of the electronicbinocular telescope 10 is minimized. The shaking in the displaydirection (display position (object position corresponding to thedisplay range, such as an object position corresponding to the centerposition in the display range)) caused by the electronic binoculartelescope 10 that is shaken in the front-back, top-bottom or left-rightdirections, while the orientation of the electronic binocular telescope10 remains the same, may be minimized. Further, the shaking in thedisplay direction caused by the shaking of the orientation of theelectronic binocular telescope 10 may be minimized. Furthermore, both ofthe above mentioned shakings may be minimized. The shaking of theelectronic binocular telescope 10 may be detected by the gyro sensor106, or may be detected by a member (sensor) that is different from thegyro sensor 106. The method for detecting the shaking of the electronicbinocular telescope 10 is not especially limited. For example, theshaking (micro-vibration) at a predetermined frequency may be detectedand shaking in the display direction caused by this shaking may beminimized. Moreover, vibration (camera shake) may be determined in acase where a value of a shaking signal (detection result) outputted fromthe vibration sensor is less than a threshold, and an orientation changethat is not vibration may be determined in a case where the value of theshaking signal is the threshold or more. The vibration and the change oforientation that is not vibration can be distinguished and detectedusing various techniques disclosed in Japanese Patent ApplicationLaid-open No. 2015-75697 or the like.

Instead of the control of the position of the display range based on theshaking of the electronic binocular telescope 10, the imaging directionof the camera 101 may be controlled. Specifically, the imaging directionmay be controlled such that the shaking in the display direction causedby the shaking of the electronic binocular telescope 10 is minimized.

In the same manner, the imaging direction may be controlled instead ofthe control of the position of the display range (movement by the movingamount Al) based on the object distance L.

The control method for moving the display range to the pan direction maybe different from the control method for moving the display range to thetilt direction. For example, moving the display range to one of the pandirection and the tilt direction may not be performed, and moving thedisplay range to the other of the pan direction or the tile directionmay be performed in accordance with the processing flow in FIG. 1 .

Embodiment 2

Embodiment 2 of the present invention will be described next. In theexample described in Embodiment 1, when the change amount of theorientation of the electronic binocular telescope 10 (head) is largerthan a predetermined amount, the processing is switched in accordancewith the result of the comparison between the change direction of theorientation of the electronic binocular telescope 10 and the changedirection of the line-of-sight, and depending on whether theline-of-sight is changed or not, whereby the moving amount A2 isdetermined. In Embodiment 2, an example of determining the moving amountA2 in accordance with the detected line-of-sight without considering theorientation of the electronic binocular telescope 10, in a case wherethe change amount of the orientation of the electronic binoculartelescope 10 is larger than a predetermined amount, will be described.In the following, aspects (configuration, processing or the like)different from Embodiment 1 will be described in detail, and aspects thesame as Embodiment 1 will be omitted unless description is necessary.

FIG. 6 is a flow chart depicting the processing flow (processing flow ofthe electronic binocular telescope 10) according to Embodiment 2, and isa modification of the flow chart in FIG. 1 . In FIG. 6 , a sameprocessing step as FIG. 1 is denoted with a same reference sign as FIG.1 . In a case where the orientation changed by a change amount largerthan the threshold (YES in S104), processing advances to S201, and toS110 after S201. In S201, the CPU 201 detects the line-of-sight of theuser using the line-of-sight-detecting unit 107, and determines themoving amount A2 to match the display direction with the detecteddirection of the line-of-sight.

As described above, in Embodiment 2 as well, the position of the displayrange is not controlled based on the detected line-of-sight if thechange amount of the detected orientation (orientation of the electronicbinocular telescope 10 (head)) is smaller than a predetermined amount.The position of the display range is controlled based on the etectedline-of-sight if the change amount of the detected orientation is largerthan the predetermined amount. Thereby the same effect as Embodiment 1can be acquired.

Embodiment 3

Embodiment 3 of the present invention will be described next. In thefollowing, aspects (configuration, processing and the like) differentfrom Embodiment 1 will be described in detail, and aspects the same asEmbodiment 1 will be omitted unless description is necessary. InEmbodiment 3, the displays 102 and 103 display an image (e.g. entireimage) captured by the camera 101.

FIG. 7 is a flow chart depicting the processing flow (processing flow ofthe electronic binocular telescope 10) according to Embodiment 3, and isa modification of the flow chart in FIG. 1 . In FIG. 7 , a sameprocessing step as FIG. 1 is denoted with a same reference sign as FIG.1 .

FIG. 8A indicates the internal direction (imaging direction immediatelyafter power is turned ON; reference direction) of the imaging directionof the camera 101. As illustrated in FIG. 8A, the reference direction isa direction where the optical axis of the camera 101 is parallel withthe front direction of the electronic binocular telescope 10 (directionin which the face of the user wearing the electronic binocular telescope10 is facing). FIG. 8A is drawn from the viewpoint viewing the head ofthe user from above, so that the pan direction component of the imagingdirection can be visually understood, but this is the same for the tiltdirection component of the imaging direction as well. In the following,only the control to change the imaging direction in the pan directionwill be described, but the imaging direction may also be changed in thetilt direction by the same method as the control method to change theimaging direction in the pan direction.

In FIG. 7 , processing steps 5701 and 5702 to S707 are performed insteadof the processing steps S102 and S106 to S111 in FIG. 1 .

In S701, the camera 101 detects (acquires) the object distance L fromthe result of the AF control in S101, and CPU 201 determines(calculates) the rotating amount B1 of the camera 101 based on thedetected object distance L.

FIG. 9 indicates the relationship between the object distance L and therotating amount B1. The star symbol in FIG. 9 indicates the observationtarget that exists in front of the user. Normally in the case of theobservation with the naked eye, the user faces the object and capturesthe object at the center of the field-of-view. Here a case where theimaging direction of the camera 101 is the reference direction will beconsidered. In this case, depending on the position at which the camera101 is installed, the observation target, which the user would captureat the center of the field-of-view of the user were viewing with theirnaked eye, may not be displayed at the center of the display, and theuser may have a sense of discomfort. The rotating amount B1 determinedin S701 is a rotating amount to reduce such a sense of discomfort. Hereit is assumed that a rotating direction to the left is a positivedirection, and a rotating direction to the right is a negativedirection. In FIG. 9 , the camera 101 is installed at a position that isshifted to the right from the center of the head by the distance a.Therefore if the camera 101 is rotated by the rotating amountB1=φ1=arctan (a/L1), the observation target existing at the objectdistance L1 (observation target existing in front of the user) can bedisplayed at the center of the display. In the same manner, if thecamera 101 is rotated by a rotating amount B1=φ2=arctan (a/L2), theobservation target existing at the object distance L2 (observationtarget existing in front of the user) can be displayed at the center ofthe display. In this way, in S701, the rotating amount B1, which islarger as the object distance is shorter, is determined based on therelationship expression “B1=arctan (a/L)” from the object distance L.According to this relationship expression, the rotating amount B1becomes virtually 0 (zero) when the object distance L is relativelylong. Therefore in a case where observation is basically performed onlyfor an object at long distance, or in a case where the object distance Lis longer than a predetermined distance, the rotating amount B1 may beset to 0 (rotating amount B1=0).

In S105 in FIG. 7 , using the line-of-sight-detecting unit 107, the CPU201 detects the line-of-sight of the user in a period when theorientation of the electronic binocular telescope 10 (head) is changing,and switches processing so that the imaging direction of the camera 101is controlled based on this line-of-sight.

FIG. 8B indicates a state where the head of the user unintentionallymoved in the state of FIG. 8A. In a case where the head movesunintentionally, the absolute line-of-sight is more likely to bemaintained in an approximate constant state. Therefore in FIG. 8B, thehead moved to the left (positive direction) by the angle θ1, but thedirection of the absolute line-of-sight remains at the originalreference direction (reference direction before the head moved;reference direction in FIG. 8A). In other words, the movement to theright (negative direction; opposite direction of the movement of thehead) by the angle θ1 is generated as a relative movement of theline-of-sight with respect to the display. When the head moves, theelectronic binocular telescope 10 moves integrally with the head.Therefore in FIG. 8B, the electronic binocular telescope 10 also movedto the left by the angle θ1. In the state in FIG. 8B, using the gyrosensor 106, the CPU 201 determines that the electronic binoculartelescope 10 (head) moved to the left by the angle θ1. Furthermore,using the line-of-sight-detecting unit 107, the CPU 201 determines thatthe line-of-sight (relative line-of-sight) moved to the right (oppositedirection of the movement of the electronic binocular telescope 10(head)) by the angle θ1.

In the case where the line-of-sight (relative line-of-sight) changed tothe opposite direction of the change direction of the orientation of theelectronic binocular telescope 10 (head) like this, processing advancesto S702 based on the determination that the user does not desire tochange the viewing direction (line-of-sight direction) and the headunintentionally moved. Then in S702, the CPU 201 determines (calculates)the rotating amount B2 to rotate the camera 101 such that theangle-of-view immediately before the detection of the movement of thehead is maintained. In other words, the CPU 201 determines the rotatingamount B2 to rotate the camera 101 such that the movement of the head iscancelled. In the state of FIG. 8B, the rotating amount B2 to rotate thecamera 101 to the right by the angle θ1 is determined. At this time, therotating amount B2 may be determined based on the detection result bythe gyro sensor 106 (angle θ1 in the left direction), or may bedetermined based on the detection result by the line-of-sight-detectingunit 107 (angle θ1 in the right direction), or may be determined basedon both of these determination results. The angle (degree) detected bythe gyro sensor 106 and the angle (degree) detected by theline-of-sight-detecting unit 107 may be different.

In the example described above, the rotating amount B2 to make thechange of the imaging direction of the camera 101 caused by the changeof the orientation of the electronic binocular telescope 10 (head) to 0(zero), is determined in S702, but the rotating amount B2 determined inS702 is not limited to this. All that is required here is to decreasethe change of the imaging direction caused by the change of theorientation, and the change of the imaging need not be exactly 0 (zero).

FIG. 8C indicates a state where the user intentionally moved their headin the state of FIG. 8A in order to change the observation target. Inthe case of intentionally moving the head, the absolute line-of-sight ismore likely to move in the same directions as the moving direction ofthe head. Therefore in FIG. 8C, an integrated movement of the head andthe absolute line-of-sight occurred. Specifically, the head moves to theleft by the angle θ1, and the absolute line-of-sight also moves to theleft by the angle θ1. In other words, the movement of the line-of-sightrelative to the display is not generated. In the state of FIG. 8C, usingthe gyro sensor 106, the CPU 201 determines that the electronicbinocular telescope 10 (head) moved to the left by the angle θ1.Furthermore, using the line-of-sight-detecting unit 107, the CPU 201determines that the line-of-sight (relative line-of-sight) is not moved.

In the case where only the change of the orientation of the electronicbinocular telescope 10 (head) is detected like this, processing advancesto S703 based on the determination that the user intentionally movedtheir head in order to change the viewing direction in accordance withthe movement of the head. Then in S703, the CPU 201 determines therotating amount B2=0 which does not rotate the camera 101. In otherwords, the CPU 201 determines the rotating amount B2 to maintain theimaging direction of the camera 101 in the line-of-sight direction(direction of the line-of-sight).

FIG. 8D indicates a state where the user intentionally moved their headin the state of FIG. 8A, in order to track (visually follow) anobservation target (moving object) that moves fast. In the case oftracking an observation target that moves fast, the absoluteline-of-sight is more like to move by a moving amount larger than themoving amount of the head (including the moving amount of the head) inthe same direction as the moving direction of the head. Therefore inFIG. 8D, the head moves to the left by the angle θ1, and the absoluteline-of-sight moves to the left by the angle θ1+θ2. In other words, therelative movement of the line-of-sight, with respect to the display, isgenerated to the left by the angle θ2. In the state of FIG. 8D, usingthe gyro sensor 106, the CPU 201 determines that the electronicbinocular telescope 10 (head) moved to the left by the angle θ1.Furthermore, using the line-of-sight-detecting unit 107, the CPU 201determines that the line-of-sight (relative line-of-sight) moved to theleft by the angle θ2.

In the case where the line-of-sight (relative line-of-sight) changed inthe same direction as the change direction of the orientation of theelectronic binocular telescope 10 (head) like this, processing advancesto S704 based on the determination that the user intentionally movedtheir head in order to radically change the viewing direction. Then inS704, the CPU 201 determines the rotating amount B2 to change theimaging direction of the camera 101 to the line-of-sight direction.

The rotating amount B2 that is determined in S704 is not limited to therotating amount to change the imaging direction to the line-of-sightdirection. All that is required here is to increase the change of theimaging direction caused by change of orientation, and the imagingdirection need not match with the line-of-sight direction.

In S705, the CPU 201 determines the rotating amount B2=0. In a casewhere the focal distance is short (in a case of a wide angle-of-view;corresponds to NO in S103), the change of the observation range (objectrange (angle-of-view) of an image displayed on the display) caused by anunintentional change of orientation (unintentional movement of the head)of the electronic binocular telescope 10, normally does not standoutvery much. In a case where the change amount of the orientation of theelectronic binocular telescope 10 (head) is small (corresponds to NO inS104), the change of the observation range caused by the unintentionalchange of the orientation of the electronic binocular telescope 10 doesnot standout very much, and the user may move the line-of-sight so as tolook out over entire observation range. Therefore in Embodiment 3,processing can advance to S705 in such cases. At this time, theline-of-sight detection is not performed, and even if the line-of-sightdetection is performed, the detection result thereof is not used. Whenthe focal distance is long (focal distance of telephoto observation;focal distance at which the change of the observation range caused byunintentional change of orientation of the electronic binoculartelescope 10 tends to standout), the processing in S103 may be omittedso that processing can advance to S104 regardless the focal distance.

In S706, the CPU 201 determines (calculates) the final rotating amountB=B1+B2 of the camera 101, based on the rotating amount B1 determined inS701 and the rotating amount B2 determined in any step of S702 to S705.In the case where it is predetermined that the rotating amount B1 is 0(B1=0), such as a case where the observation is basically performed onthe objects at long distance, the processing in S701 may be omitted sothat the rotating amount B=B2 is determined regardless the objectdistance L.

In S707, the CPU 201 rotates the camera 101 by the rotating amount Bdetermined in S706, using the camera-rotating unit 202.

As described above, in Embodiment 3, the imaging direction of the camera101 is not controlled based on the detected line-of-sight if the changeamount of the detected orientation (orientation of the electronicbinocular telescope 10 (head)) is smaller than a predetermined amount.The imaging direction is controlled based on the detected line-of-sightif the change amount of the detected orientation is larger than thepredetermined amount. Thereby an unintended change of the observationrange can be minimized, and a sensation close to the observation withthe naked eye (sensation with no or minimal sense of discomfort) can beprovided to the user.

Besides the control of the imaging direction in the processing flow inFIG. 7 , the imaging direction may be constantly controlled based on theshaking of the electronic binocular telescope 10, such as camera shakecorrection. Specifically, the imaging direction may be constantlycontrolled so that the shaking in the imaging direction caused by theshaking of the electronic binocular telescope 10 is minimized. Theshaking in the imaging direction (imaging position (object positioncorresponding to the imaging range, such as an object positioncorresponding to the center position in the imaging range; optical axisposition of the camera 101) caused by the electronic binocular telescope10 that is shaken in the front-back, top-bottom, or left-rightdirections, while the orientation of the electronic binocular telescope10 remains the same, may be minimized. Further, the shaking in theimaging direction caused by the shaking of the orientation of theelectronic binocular telescope 10 may be minimized, or both of the abovementioned shakings may be minimized. The shaking of the electronicbinocular telescope 10 may be detected by the gyro sensor 106, or may bedetected by a member (sensor) that is different from the gyro sensor106. The method for detecting the shaking of the electronic binoculartelescope 10 is not especially limited. For example, the shaking(micro-vibration) at a predetermined frequency may be detected, andshaking in the imaging direction caused by this shaking may beminimized. Moreover, vibration (camera shake) may be determined om acase where a value of a shaking signal (detection result) outputted fromthe vibration sensor is less than a threshold, and the orientationchange that is not vibration may be determined when the value of theshaking signal is the threshold or more. The vibration and the change ofthe orientation that is not vibration can be distinguished and detectedusing various techniques disclosed in Japanese Patent ApplicationLaid-open No. 2015-75697 or the like.

A possible configuration here is that a part of the captured image isdisplayed on the display as a display range. In this case, instead ofthe control of the imaging direction based on the shaking of theelectronic binocular telescope 10, the position of the display range maybe controlled. Specifically, the position of the display range may becontrolled such the shaking in the display directions caused by theshaking of the electronic binocular telescope 10 (direction from thecamera 101 to the object corresponding to the display range, such as adirection from the camera 101 to an object position corresponding to thecenter position of the display range) is minimized.

In the same manner, in the case of displaying a part of the capturedimage on the display as the display range, the position of the displayrange may be controlled, instead of the control of the imaging direction(rotation by the rotating amount B1) based on the object distance L.

The control method for changing the imaging direction to the pandirection may be different from the control method for changing theimaging direction to the tilt direction. For example, a change of theimaging direction to one of the pan dire direction and the tiltdirection may not be performed, and a change of the imaging direction tothe other of the pan direction and the tilt direction may be performedin accordance with the processing flow in FIG. 7 .

Embodiment 4

Embodiment 4 of the present invention will be described next. In theexample described in Embodiment 3, when the change amount of theorientation of the electronic binocular telescope 10 (head) is largerthan a predetermined amount, the processing is switched in accordancewith the result of the comparison between the change direction of theorientation of the electronic binocular telescope 10 and the changedirection of the line-of-sight, and depending on whether theline-of-sight changed or not, whereby the rotating amount B2 isdetermined. In Embodiment 4, an example of determining the rotatingamount B2 in accordance with the detected line-of-sight withoutconsidering the orientation of the electronic binocular telescope 10 ina case where the change amount of the orientation of the electronicbinocular telescope 10 is larger than a predetermined amount, will bedescribed. In the following, aspects (configuration, processing and thelike) different from Embodiment 3 will be described in detail, andaspect the same as Embodiment 3 will be omitted unless description isnecessary.

FIG. 10 is a flow chart depicting the processing flow (processing flowof the electronic binocular telescope 10) according to Embodiment 4, andis a modification of the flow chart in FIG. 7 . In FIG. 10 , a sameprocessing step as FIG. 7 is denoted with the same reference sign asFIG. 7 . In a case where the orientation changed by a change amountlarger than the threshold (YES in S104), processing advances to S1001,and to S706 after S1001. In S1001, the CPU 201 detects the line-of-sightof the user using the line-of-sight-detecting unit 107, and determinesthe rotating amount B2 to match the imaging direction with the detecteddirection of the line-of-sight.

As described above, in Embodiment 4 as well, the imaging direction ofthe camera 101 is not controlled based on the detected line-of-sight ifthe change amount of the detected orientation (orientation of theelectronic binocular telescope 10 (head)) is smaller than apredetermined amount. The imaging direction is controlled based on thedetected line-of-sight if the change amount of the detected orientationis larger than the predetermined amount. Thereby the same effect asEmbodiment 3 can be acquired.

Embodiments 1 to 4 (including the modifications) are merely examples,and the configurations acquired by appropriately modifying and changingthe configurations of Embodiments 1 to 4 within the scope of the spiritof the present invention are included in the present invention. Theconfigurations acquired by appropriately combining the configurations ofthe Embodiments 1 to 4 are also included in the present invention.

According to the present disclosure, an unintentional change of theobservation range can be minimized, and a sensation close to theobservation with the naked eye (sensation with no or minimal sense ofdiscomfort) can be provided to the user.

Other Embodiments

Embodiment(s) of the present invention can also be realized by acomputer of a system or apparatus that reads out and executes computerexecutable instructions (e.g., one or more programs) recorded on astorage medium (which may also be referred to more fully as a‘non-transitory computer-readable storage medium’) to perform thefunctions of one or more of the above-described embodiment(s) and/orthat includes one or more circuits (e.g., application specificintegrated circuit (ASIC)) for performing the functions of one or moreof the above-described embodiment(s), and by a method performed by thecomputer of the system or apparatus by, for example, reading out andexecuting the computer executable instructions from the storage mediumto perform the functions of one or more of the above-describedembodiment(s) and/or controlling the one or more circuits to perform thefunctions of one or more of the above-described embodiment(s). Thecomputer may comprise one or more processors (e.g., central processingunit (CPU), micro processing unit (MPU)) and may include a network ofseparate computers or separate processors to read out and execute thecomputer executable instructions. The computer executable instructionsmay be provided to the computer, for example, from a network or thestorage medium. The storage medium may include, for example, one or moreof a hard disk, a random-access memory (RAM), a read only memory (ROM),a storage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or B1u-ray Disc(BD)TM), a flash memory device, a memory card, and the like.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

1. A display device configured to be used in a state of being fixed to ahead of a user, comprising: a camera; a display configured to display apart of an image captured by the camera as a display range; anorientation detection sensor configured to detect an orientation of thedisplay device; a line-of-sight detection sensor configured to detect aline-of-sight of the user to the display; and a processor configured notto control a position of the display range based on the line-of-sightdetected by the line-of-sight detection sensor in a case where a changeamount of the orientation detected by the orientation detection sensoris smaller than a predetermined amount, and to control the position ofthe display range based on the line-of-sight in a case where the changeamount of the orientation is larger than the predetermined amount. 2.The display device according to claim 1, wherein in a case where thechange amount of the orientation is larger than the predeterminedamount, if the line-of-sight changes in an opposite direction of achange direction of the orientation, the processor controls the positionof the display range such that a change of a display direction caused bya change of the orientation decreases.
 3. The display device accordingto claim 1, wherein in a case where the change amount of the orientationis larger than the predetermined amount, if the line-of-sight changed ina same direction as a change direction of the orientation, the processorcontrols the position of the display range such that a change of adisplay direction caused by a change of the orientation increases. 4.The display device according to claim 1, wherein in a case where thechange amount of the orientation is larger than the predeterminedamount, the processor controls the position of the display range suchthat a display direction matches with a direction of the line-of-sight.5. The display device according to claim 1, wherein a focal distance ofthe camera is changeable, and in a case where the focal distance isshorter than a predetermined distance, the processor does not controlthe position of the display range based on a detection result by theorientation detection sensor and a detection result by the line-of-sightdetection sensor.
 6. The display device according to claim 1, whereinthe camera additionally detects an object distance, and the processoradditionally controls the position of the display range based on thedetected object distance.
 7. The display device according to claim 1,wherein the processor additionally controls the position of the displayrange based on a shaking of the display device.
 8. The display deviceaccording to claim 1, wherein the display range is moveable in a pandirection and a tilt direction independently.
 9. The display deviceaccording to claim 8, wherein a control method for moving the displayrange in the pan direction is different from a control method for movingthe display range in the tilt direction.
 10. The display deviceaccording to claim 1, wherein the display device is a display devicewhich is used while the user fixes the display device to the head of theuser, or a display device which is wearable on the head of the user. 11.A control method of a display device including a camera and a displayconfigured to display a part of an image captured by the camera as adisplay range, the display device being configured to be used in a stateof being fixed to a head of a user, the control method comprising: anorientation detection step of detecting an orientation of the displaydevice; a line-of-sight detection step of detecting a line-of-sight ofthe user to the display; and a control step of not controlling aposition of the display range based on the line-of-sight detected in theline-of-sight detection step in a case where a change amount of theorientation detected in the orientation detection step is smaller than apredetermined amount, and controlling the position of the display rangebased on the line-of-sight in a case where the change amount of theorientation is larger than the predetermined amount.
 12. Anon-transitory computer readable storage medium that stores a program,wherein the program causes a computer to execute a control method of adisplay device including a camera and a display configured to display apart of an image captured by the camera as a display range, the displaydevice being configured to be used in a state of being fixed to a headof a user, the control method comprising: an orientation detection stepof detecting an orientation of the display device; a line-of-sightdetection step of detecting a line-of-sight of the user to the display;and a control step of not controlling a position of the display rangebased on the line-of-sight detected in the line-of-sight detection stepin a case where a change amount of the orientation detected in theorientation detection step is smaller than a predetermined amount, andcontrolling the position of the display range based on the line-of-sightin a case where the change amount of the orientation is larger than thepredetermined amount.